The invention relates generally to data analysis, and relates more particularly to data analysis techniques using nonlinear regression.
Nonlinear regression refers to the development of empirical models often used in data analysis to encapsulate temporal and/or structural relationships between observed quantities (“input variables”) and quantities of interest (“output variables” or “target variables”), which may be difficult to observe directly. Specifically, the goal of nonlinear regression is to construct a mathematical model that is capable of accurately estimating an unobserved target variable as a function of the settings of the collection of input variables to particular input states or patterns.
Typically, these mathematical models are produced by applying machine learning or training techniques to a data set that contains a number of historical exemplars, where each exemplar i comprises a particular input pattern {right arrow over (x)}i (with each of the input variables set to a particular value) and an associated target value yi that was observed or known by some means. Training on the data set aims to obtain a general functional mapping ŷ=F({right arrow over (x)}) that estimates a predicted or likely target value ŷ for a general input pattern {right arrow over (x)}. A desirable property of the mapping is that it is general enough to provide accurate target value estimates for input patterns not contained in the training data set.
Existing techniques for performing nonlinear regression (including neural networks, regression trees, splines, wavelets, and support vector regression, among others) commonly suffer from a limitation referred to as the curse of dimensionality. That is, it becomes progressively (e.g., exponentially) more difficult to learn an accurate functional mapping as the dimensionality (number of features or state variables) of the input space increases.
Thus, there is a need for an improved method for regression modeling that addresses the curse of dimensionality which limits existing methods.